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1  Introduction 

The  problem  of  how  to  compute  efficiently  estimates  of 
the  optical  flow  at  sparse  locations  is  of  critical  impor¬ 
tance  for  practical  implementations  in  a  number  of  differ¬ 
ent  tasks.  A  specific  example  is  the  detection  of  expan¬ 
sion  of  the  visueil  field  with  a  rough  estimate  of  time-io- 
crash  (TTC).  The  question  has  also  interesting  relations 
with  biology,  as  we  will  discuss  later.  In  this  paper  we 
propose  an  efficient  algorithm  for  computing  the  opti¬ 
cal  flow  which  performs  well  in  a  number  of  experiments 
with  sequences  of  real  images  and  is  well  suited  to  a  VLSI 
implementation. 

Optical  flow  algorithms  based  on  patchwise  correla¬ 
tion  of  filtered  images  perform  in  a  satisfactory  way  [3] 
and  better  in  practice  than  most  other  approaches  (see 
[1]).  Their  main  drawback  is  computational  complexity 
that  forbid  at  present  useful  VLSI  implementations.  In 
this  paper  we  show  that  ID  patchwise  correlation  may 
provide  a  sufificiently  accurate  estimate  of  the  optical 
flow^.  We  will  then  show  with  experiments  on  real  im¬ 
age  sequences  how  to  apply  this  technique  to  measure 
time-to-crash,  by  exploiting  a  recently  proposed  scheme 
[5].  The  latter  scheme,  which  is  robust  and  invariant 
to  the  position  of  the  focus  of  expansion  or  the  center 
of  rotation,  relies  on  sparse  measurements  of  either  the 
normal  or  the  tangential  component  of  the  optical  flow 
(relative  to  a  closed  contour).  We  will  also  discuss  some 
broad  implications  of  this  work  for  the  practical  compu¬ 
tation  of  the  optical  flow  and  for  biology,  in  particular 
its  relation  to  Reichardt’s-type  models. 

There  are  two  main  and  quite  separate  contributions 
in  this  paper; 

1.  an  efficient  ID  correlation  scheme  to  estimate  the 
optical  flow  along  a  desired  direction 

2.  the  experimental  demonstration  that  a  previously 
proposed  algorithm  for  estimating  time-to-crash 
performs  satisfactorily  in  a  series  of  experiments 
with  real  images  in  which  the  elementary  measure¬ 
ments  of  the  flow  are  obtained  by  the  new  ID  cor¬ 
relation  scheme. 

2  Computing  the  Optical  Flow  along  a 
Direction 

How  can  the  component  of  the  optical  flow  be  measured 
efficiently  along  a  certain  desired  direction?  As  argued 
by  Verri  and  Poggio  [12]  a  qualitative  estimate  is  often 
suflBcient  for  many  visual  tasks.  For  the  task  of  detect¬ 
ing  a  potential  crash,  for  instance,  it  has  been  suggested 
([5])  that  a  precise  measurement  of  the  normal  compo¬ 
nent  of  the  flow  may  not  be  necessary,  since  the  precise 

*In  this  paper  we  use  mainiy  the  distance  rather  than 
the  correlation  itself.  Since  the  two  measures  are  equivalent 
for  the  purposes  of  this  paper,  we  will  often  use  the  terms 
“correlation”  and  “distance”  in  an  interchangeable  way. 


definition  of  the  optical  flow  is  itself  somewhat  arbitrary: 
it  is  sufficient  that  the  estimate  be  quahtatively  consis¬ 
tent  with  the  values  of  the  perspective  2D  projection 
of  the  “true”  3D  velocity  field  for  the  particular  stim¬ 
ulus.  In  other  words,  even  estimates  that  don’t  really 
measure  image-plane  velocity  (like  Reichardt’s  correla¬ 
tion  model  or  equivalent  energy  models),  since  they  also 
depend  on  spatial  structure  of  the  image,  may  be  ac¬ 
ceptable  for  several  visual  tasks,  if  their  estimates  are 
consistent  over  the  visual  field.  Certain  uses  of  a  crash 
detector  are  good  examples.  It  turns  out  that  even  a 
rough  estimate  of  time-to-crash  (TTC)  is  possible  using 
approximate  estimates  of  the  optical  flow  field.  Flies  and 
other  insects  rely  for  landing  on  what  appears  to  be  a 
qualitative  estimate  of  the  time-to-crash! 

2.1  ID  correlation  of  2D  patches 

.*  possible  approach  for  an  approximative  estimate  of 
the  optical  flow  is  to  use  a  ID  correlation  scheme  be¬ 
tween  two  successive  frames,  instead  of  2D  correlation, 
as  in  [3].  The  basic  idea  underlying  the  full  2D  correla¬ 
tion  technique  that  we  label  2D-2D  in  this  paper^  is  to 
measure,  for  each  desired  location,  the  (x,y)  shift  that 
maximizes  the  correlation  between  2D  patches  centered 
around  the  desired  location  in  successive  frames.  The 
patchwise  correlation  between  the  image  at  time  t  and 
at  time  t  +  6t  is  defined  as 

^{6x,6y;t)  =  r"<S>l  =  j  r(^,r);t)I{Sx+^,6y+T);t-\-6l)cl^dT) 

(1) 

where  /“'(^,  rj;  t)  is  the  image  at  time  t  windowed  to  the 
patch  of  interest  and  set  to  0  outside  it.  The  L2  distance 
has  very  similar  properties  to  the  correlation  measure 
In  the  context  of  this  paper,  minimizing  the  L2  distance 
is  exactly  equivalent  to  maximizing  the  correlation  (the 
observation  is  due  to  F.  Girosi).  As  noticed  before  [3], 
the  previous  idea  can  be  regarded  as  an  approximation 
of  a  regularization  solution  to  the  problem  of  computing 
the  opticed  flow**.  Usually,  one  does  not  use  grey  values 
directly  but  rather  some  filtered  version  of  the  image, 
for  instance  through  a  Laplacian-of-a-Gaussian  filter  (see 
[3]),  possibly  at  different  resolutions. 

Let  us  call  D(6x,Sy)  the  L2  distance  between  2 
patches  in  2  frames  at  location  (x,  y)  as  a  function  of  the 
shift  vector  (Sx,Sy).  The  “winner-take-all”  scheme  finds 
s*  =  (6J,^*)  that  minimizes  D  (or  maximize  the  corre¬ 
lation  function  ^(^r,^y))  and  assumes  that  the  optical 

^It  is  also  called  winner-take-all  method. 

^The  L2  distance  is  in  this  case  the  square  root  of  the 
sum  of  the  squares  of  the  differences  between  values  of  cor¬ 
responding  pixels.  Other  “robust”  distance  metric  may  be 
used,  such  as  the  sum  of  absolute  values. 

^And  in  turn  several  definitions  of  the  optical  flow  such 
as  Horn  and  Schunk’s,  can  be  shown  to  be  approximations  of 
the  correlation  technique  [6]. 


the 


Figure  1;  The  search  space  for  the  ID  —  2D  scheme  used 
for  the  computation  of  the  x  and  y  components  of  the 
optical  flow. 

flow  estimate  is  u*  =  s*/A/,  where  At  is  the  inter&ame 
interval. 

It  is  natural  to  consider  whether  the  component  of 
u*  along  a  given  direction,  for  instance  z,  may  be  es¬ 
timated  in  a  satisfactory  way  simply  by  computing  the 
Sx  that  minimizes  D{Sx,0),  that  is  the  patchwise  cor¬ 
relation  as  a  function  of  z  shifts  only.  We  have  found 
in  our  experiments  that  ID  correlation  of  a  2D  patch 
provides  estimates  of  6x*  that  are  very  close  to  the  es¬ 
timates  obtained  from  the  2D-2D  technique.  We  label 
this  technique  1D-2D,  since  it  involves  one-dimensional 
correlations  on  2D  patches. 

If  we  combine  horizontal  and  vertical  motion  detectors 
of  our  ID,  winner-take-all  type  (see  fig-l),  we  obtain  an 
appealing  scheme  to  estimate  the  optical  flow  field  at 
one  point.  The  optical  flow  in  one  point  is  the  vector 
sum  of  the  z  and  y  components  computed  by  using  such 
motion  detectors.  The  key  aspect  of  this  approach  is  its 
reduction  of  the  complexity  of  the  problem,  while  main¬ 
taining  a  good  estimation  of  the  flow  field:  a  complete 
two-dimensional  search  required  in  the  winner-take-all 
scheme  [3]  is  reduced  to  two  one-dimensional  searches. 
Let  us  call  Umor  the  maximum  velocity  expected  on  the 
image  plane.  In  [3]  the  search  space  size  to  scan  is 
(2vmax  +  1)^  for  each  point;  in  our  approach,  its  size 
is  limited  to  2{2vmax  +  1). 

2.2  ID  correlation  of  ID  patches 

So  far  we  have  discussed  that  ID  correlation  of  2D 
patches  gives  a  satisfactory  estimate  of  the  optical  flow 
between  two  successive  frames,  reducing  the  search  space 
of  corresponding  points.  This  is  equivalent  to  saying  that 


min6g^{6x,  0) 

and 

Tniniy^{Q,6y) 

give  a  satisfactory  estimate  of 

Tninst,6y^{6x,6y). 

This  suggests  a  further  simplification;  instead  of  $(6z,  0) 
consider  a  projection  on  z  of  ^{6x,  Sy)  obtained  by  some 
form  of  averaging  operation  on  y,  that  is 

0  ♦  /l2 

where  A2  is  a  2D  filter  such  as  a  Gaussian  elongated  in 
ihe  y  direction  and  *  stands  tor  the  convolution  operator. 
By  well  known  properties  of  the  Gaussian  function,  ^2 
can  always  be  written  as 

/i2  =  h  *  h, 

where  h  are  Gaussian  functions  of  appropriate  variance. 
Assuming  that  we  can  neglet  the  patch  size  in  the  defi¬ 
nition  of  we  can  write; 

$  ♦  /»2  =  (/t  ♦  h)  0  {It+Si  *  h)  (2) 

where  1%  —  I(x,y,t). 

Thus,  in  the  approximation  of  a  large  patch  size,  pro¬ 
jecting  the  correlation  function  is  equivalent  to  appropri¬ 
ately  filtering  the  two  images  before  correlation.  Since 
it  is  usually  better  to  discount  the  average  intensity  as 
well  as  small  gradients  through  a  high-pass  filtering  op¬ 
eration,  in  order  to  estimate  the  z-component  of  u,  we 
just  perform  a  Gaussian  smoothing  in  the  y  direction,  as 
shown  in  eq.  2,  and  then  perform  an  additional  convo¬ 
lution  with  the  first  or  second  derivative  of  a  Gaussian 
function  elongated  in  the  z  direction.  Therefore  the  in¬ 
tensity  function  that  is  used  in  practice  in  the  correlation 
operation  is: 

4  =  (G'„,(y)W,)*G",(z)  (3) 

where  <Xz  and  <Ty  define  the  receptive  field  of  such  an 
elementary  motion  detector.  After  this  filtering  step,  it 
is  sufficient  to  evaluate  the  maximum  of  the  correlation 
function  only  on  ID  patches  to  obtain  an  estimate  of  the 
z  component  of  the  flow.  The  previous  argument  does 
not  strictly  apply  to  the  L2  distance  measure  that  we 
have  used  in  our  experiments.  The  very  close  similarity 
between  correlation  and  distance,  however,  suggests  a 
very  similar  behavior  in  both  cases.  We  label  this  tech¬ 
nique  the  7D-iD  scheme  since  it  involves  ID  correlations 
of  ID  patches. 


Figure  2:  A  TTC  detector  consisting  of  elementary  mo¬ 
tion  detectors  (see  figure  1)  at  several  locations  along  a 
closed  contour.  Each  of  the  elementary  motion  detec¬ 
tors  could  be  replaced  by  a  single  detector  normal  to  the 
circle. 

3  A  crash  detector:  the  Green  theorem 
scheme 

As  described  in  [5]  (see  abo  [2]),  the  divergence  of  the 
optical  field  Vu(x,  y)  is  a  differential  measure  of  the  local 
expansion  (Vu(i,  y)  =  For  a  linear 

field  (i.e.  u(x)  =  Ax),  the  divergence  of  u  b  the  same 
everywhere.  In  the  case  of  linear  fields  (and  all  fields  can 
be  approximated  by  linear  fields  close  to  the  singularity), 
the  integral  of  the  divergence  over  an  area  is  invariant 
with  respect  to  the  position  of  the  center  of  expansion. 
Green’s  theorems  show  that  the  integral  over  a  surface 
patch  S  of  the  divergence  of  a  field  u  is  equal  to  the 
integral  along  the  patch  boundary  of  the  component  of 
the  field  which  b  normal  to  the  boundary  (u  •  n).  In 
formula 

J  Vu(x,y)dxdy  =  J  \i  udl.  (4) 

Therefore,  since  for  a  linear  field  Vu  =  2! t  where  r 
is  the  time  to  crash  (TTC),  a  TTC  detector  that  ex¬ 
ploits  the  Green  theorem  just  needs  to  sum  over  a  closed 
contour,  say  a  circle,  the  normal  component  of  the  flow 
measured  at  n  points  along  the  contour.  We  assume 
that  the  task  b  to  compute  time  to  crash  (TTC)  for 
pure  translational  motion.  Possibly  the  simplest  TTC 
detector  of  this  type,  shown  in  figure  3,  b  composed  of 
just  4  elementary  motion  detectors.  In  this  case  we  have 
to  sum  the  x-component  of  u  for  the  horbontal  detectors 
and  the  y-component  of  u  for  the  vertical  ones,  with  the 


Figure  3;  Time-to-crash  detector  that  exploits  Green 
theorem. 


correct  sign. 

Due  to  the  invariance  with  respect  to  the  position  of 
the  focus  of  expansion  (or  contraction)  we  can  in  princi¬ 
ple  arrange  a  certain  number  of  them  (see  fig.4)  on  the 
image  plane.  Our  simulations  suggest  that  one  detec¬ 
tor  with  a  large  radius  (fig.  3)  b  better  than  several, 
“smaller”  detectors  (fig.  4)  in  situations  in  which  the 
whole  vbued  field  expands,  probably  because  of  better 
numerical  stability  of  the  estimates.  Of  course,  a  “large” 
detector  has  a  poorer  spatial  resolution  and  thb  may  be 
a  problem  in  some  applications  (but  not  ours). 

We  have  discussed  so  far  schemes  for  detecting  expan¬ 
sion.  Similar  arguments  hold  for  rotation.  The  Green ’s 
theorem  relevant  to  thb  case  is  usually  called  Stokes’ 
theorem  and  takes  the  form 

J  VAu(x,y)  dS  =  J  u  dr  (5) 

which  says  that  the  total  flux  of  the  differential  measure 
of  “rotationeility”  of  the  field  V  A  u  across  the  surface 
patch  5  b  equal  to  the  integral  along  the  boundary  C 
of  the  surface  patch  of  the  component  of  the  field  which 
is  tangential  to  the  boundary.  As  described  in  [5],  each 
elementary  detector  evaluates  the  tangential  flow  com¬ 
ponent  at  the  contour  of  the  receptive  field  (see  fig.5). 
In  this  case  a  detector  has  to  compute  the  component  of 
u  along  the  tangential  direction  at  the  contour. 

4  Experimental  results 

4.1  The  1D-2D  scheme 

We  have  extensively  tested  our  approach  on  real  image 
sequences.  Each  sequence  W2is  acquired  from  a  camera 


Figure  4;  A  possible  arrangement  of  TTC  detectors  in 
the  image  plane  that  is  not  as  efficient  as  a  single  TTC 
detector  with  greater  radius  but  has  higher  spatial  reso¬ 
lution. 


Figure  5:  Motion  detector  that  exploits  Stokes’  theorem. 


mounted  on  mobile  platform  moving  at  constant  velocity. 
In  all  experiments  the  movement  of  the  vehicle  was  a 
forwcird  translation  along  a  straight  trajectory.  We  have 
verified  the  results  obtained  from  our  1D-2D  approach 
with  the  standard  winner-take-all  {2D-2D)  scheme  [3] 
[1]. 

Figure  9  shows  the  first  and  last  image  of  a  sequence 
composed  of  100  frames.  Each  image  of  the  sequence  is 
first  convolved  with  a  Gaussian  filter  having  <t  =  0.5.  In 
both  the  algorithms  we  have  used  Vmax  =  9  and  i/  =  20 
pixels,  where  v^az  is  the  maximum  expected  velocity  of 
the  points  on  the  image  plane  and  u  is  the  ray  of  the 
patch  used  for  the  evaluation  of  $.  In  other  words,  the 
correlation  window  used  for  the  optical  flow  computation 
is  41  X  41  pixels  and  the  search  space  used  is  19  x  19 
by  2D-2D  and  19  -t-  19  by  1D-2D.  Figures  10  shows  the 
optical  flows  computed  by  the  two  methods  using  two 
successive  images  of  the  sequence.  The  position  of  the 
focus  of  expansion  was  computed  by  using  the  approach 
described  in  [11]. 

We  have  used  the  method  described  in  [1 1]  and  [5]  to 
verify  the  TTC  estimation.  To  compute  the  TTC  at  a 
point  by  using  the  method  in  [11],  we  used  an  area  of 
81  X  81  pixels  around  that  point.  The  points  were  10 
pixels  apart.  To  compute  TTC  by  using  the  method  de¬ 
scribed  in  [5] ,  we  used  a  lattice  of  overlapping  motion  de¬ 
tectors.  The  distance  between  two  points  on  the  lattice 
was  10  pixels.  Each  detectors  had  a  receptive  field  of  ray 
r  =  40  pixels.  In  fig.  11,  we  compare  the  results  obtained 
by  using  the  2D-2D  estimation  of  the  optical  flow  with 
the  1D-2D  one,  by  using  the  two  different  methods  in  the 
first  stage  of  the  TTC.  Performing  a  linear  best  fit  on  the 
TTC  measurements,  we  obtain  a  slope  of  m  =  —1.036 
by  using  the  optical  flows  computed  by  2D- 2D  and  the 
method  described  in  [11],  and  m  =  —1.139  by  using  the 
optical  flows  computed  by  1D-2D  and  the  method  de¬ 
scribed  in  [5].  Comparing  the  true  TTC  (straight  line 
in  fig.  11)  with  the  TTC  measures  obtained  by  using 
the  second  method,  we  estimate  an  absolute  error  in  the 
mean  of  2.63,  with  a  standard  deviation  of  3.35  frame 
unit.  In  terms  of  relative  units,  the  error  in  the  mean  is 
5.7%  with  a  standard  deviation  of  6.1%. 

4.2  The  ID-ID  scheme 

In  this  section  we  compare  the  results  obtained  by  us¬ 
ing  2D-2D  and  ID- ID.  In  both  techniques,  we  have  used 
Vmax  =  9  and  i/  =  20  pixels.  In  other  words,  the  cor¬ 
relation  window  used  for  the  optical  flow  estimation  is 
41  x  41  pixels  for  2D-2D  and  41  pixels  for  ID-ID.  In 
the  filtering  step  we  have  used  <Tj,  =  6  and  o’*  =  3  pix¬ 
els  for  computing  the  jc-component  of  the  optical  flow. 
These  values  of  a  produce  a  receptive  field  of  an  elemen¬ 
tary  motion  detector  equals  to  that  used  by  2D-2D  (1681 
pixels).  The  fig.  (6)  shows  a  plot  of  the  2D  correlation 
function  used  in  2D-2D  over  a  2D  search  space  and  a 
2D  integration  area.  Figures  (7)  and  (8)  show  a  plot  of 
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Figure  6;  2D  distance  function.  The  arrows  indicate  the 
position  of  the  minimum. 
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ID  correlation  functions  used  by  the  ID-ID  technique 
to  estimate  both  components  of  the  optical  flow.  In  this 
case  we  used  two  ID  search  spaces  (in  x  and  y  directions 
respectively)  Jind  a  ID  integration  area.  Notice  that  this 
approach  is  capable  of  computing  a  reliable  estimation 
of  the  flow  vectors,  while  reducing  the  complexity  of  the 
problem. 

Figures  (13),  (17),  (21)  show  the  first  and  the  last  im¬ 
age  of  three  sequences  acquired  from  a  camera  mounted 
on  a  mobile  platform  moving  at  constemt  velocity,  along 
a  straight  trajectory.  Figures  (14),  (18),  (22)  show  the 
optical  flows  computed  by  the  two  methods,  by  using 
two  successive  frames  of  the  sequences.  The  mean  (con¬ 
tinuous  line)  and  the  standard  deviation  (dashed  line) 
of  the  error  on  the  optical  flow  estimation  is  shown  in 
figures  (15),  (19),  (23).  Figures  (16),  (20),  (24)  show 
the  TTC  estimation  by  using  the  two  different  methods. 
In  each  experiments,  we  have  used  only  one  TTC  detec¬ 
tor,  with  receptive  field  of  r  =  80  pixel,  composed  by  32 
elementary  motion  detectors  (see  fig.  (2)). 

5  Conclusions 

5.1  Extensions  of  the  optical  flow  algorithm 
There  are  several  directions  in  which  we  plan  to  improve 
and  extend  our  scheme: 

•  it  may  be  possible  to  reduce  further  the  number  of 
sample  points  for  Dp  (i.e.  the  number  of  shifts)  by 
using  techniques  for  learning  from  examples  such 
as  the  RBF  technique  ([4])  to  approximate  Dp{6x) 
as  Dp(Sx)  =  Y^Cr,G{x  —  t„),  and  then  find  the 
minimum  of  Dp  in  terms  of  the  dynamical  sys¬ 
tem  dx/dt  —  —(^CnG'{x  —  t„).  An  alternative 
strategy  is  to  try  to  learn  directly  the  function 


Figure  7:  ID  distance  function  computed  on  the  x  di¬ 
rection  by  using  ID- ID. 
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Figure  8:  ID  distance  function  computed  on  the  y  direc¬ 
tion  by  using  ID- ID. 


TninDp{x)  from  the  samples  of  Dp,  using  a  few  ex¬ 
amples  of  Dp  “typical”  for  the  specific  situation. 
The  conjecture  is  that  the  RBF  technique  may  be 
able  to  learn  the  mapping  minDp{x)  from  examples 
of  functions  of  the  same  class  (compare  Poggio  and 
Vetter,  1992).  A  similar  idea  is  to  try  to  learn  how 
to  sample  the  correlation  function  as  a  function  of 
past  sampled  values.  Ageiin,  the  treiining  examples 
would  be  functions  of  the  same  class.  This  would 
provide  at  each  t  am  estimate  of  the  most  appropri¬ 
ate  correlation  shifts  to  try. 

•  instead  of  simply  measuring  Di  i^\,  that  is  the 
distance  between  frame  i  and  frame  i  —  1,  we 
could  measure  in  addition  also  A,i-2,  Di  i_^,  ...and 
combine  them  in  an  estimate  of  the  optical  flow 
component  by  taking  the  average  of 
A,i-2/2Ai,  A,,_3/3A/,  etc.  This  technique  may 
be  improved  further  by  using  a  Kalman  filter. 

•  the  same  basic  scheme  of  figure  1  may  be  used  to 
compute  horizontal  and  vertical  disparities  among 
the  two  frames  of  a  stereo  pair. 

•  confidence  measures  will  be  developed  to  further 
improve  the  performance  of  the  technique. 

5.2  Biological  implications  of  our  ID  technique 

Poggio  et  al.  ([5])  conjectured  that  ’’the  specific  type  of 
elementary  motion  detectors  that  are  used  to  provide  the 
estimate  of  the  normal  component  of  the  flow  is  probably 
not  critical.  Radially  oriented  (for  expansion  and  con¬ 
traction),  two  input  elementary  motion  detectors  such 
as  the  correlation  model  [8,  9,  7,  10]  -  or  approximations 
of  it  are  likely  to  be  adequate.  The  critical  property  is 
that  they  should  measure  motion  with  the  correct  sign.” 
Our  results  confirming  their  conjecture  (since  they  sug¬ 
gest  that  ID  correlation  (or  L2  distance  estimation)  are 
sufficient  for  an  adequate  estimate  of  qualitative  proper¬ 
ties  of  the  optical  flow)  have  interesting  implications  for 
biology.  Consider  a  2D  array  of  Reichardt’s  detectors 
(for  motion  in  the  x  direction)  with  spacing  Ax  and  also 
detectors  with  spacings  2 Ax  etc.  Take  the  sum  of  all 
detectors  with  the  same  spacing  over  a  2D  patch.  Per¬ 
form  a  winner-take-all  operation  on  these  sums.  Select 
the  set  with  optimal  spacing  as  the  one  corresponding 
to  the  present  estimate  of  optical  flow.  This  scheme  is 
analog  in  time  but  otherwise  equivalent  to  the  one  we 
have  implemented.  In  formulae 

^(7,(t)  -  /.+*(<  -  M)f 

where  At  is  the  interframe  interval  in  our  implementa¬ 
tion  and  is  the  delay  in  Reichardt’s  model®,  k  represents 

®We  have  written  here  the  quadratic  version  of  Reichardt’s 
model;  the  same  argument  carries  over  to  the  standard  model 
with  multiplication:  for  the  basic  equivalence  of  the  the 
quadratic  and  multiplication  version  see  [7]) 


the  shift  in  our  computation  of  D  and  represents  the  sep¬ 
aration  between  the  inputs  to  Reichardt’s  modules,  /,(!) 
is  the  image  value  (in  general  spatially  and  temporally 
filtered)  at  location  i  and  time  t  and  the  sum  ^  is  taken 
over  the  2D  patch  of  detectors  of  the  same  type. 

Thus  an  array  of  Reichardt’s  models  with  different 
spacings  of  the  2  inputs  (in  x)  could  be  used  in  a  plau¬ 
sible  way  to  estimate  the  optical  flow  component  along 
the  direction  of  the  two-inputs  detectors.  Notice  that 
a  plausible  implementation  in  terms  of  Reichardt’s  de¬ 
tectors  of  the  2D  correlation  b2ised  algorithm  would  be 
much  harder,  since  it  would  effectively  require  detectors 
with  all  possible  2D  spacings.  This  seems  implausible 
and  contrairy  to  experimental  evidence  in  the  fly,  where 
only  a  small  number  of  separations  and  directions  (as 
small  as  3)  seem  present. 

The  above  description  is  equivalent  to  our  1D-2D 
scheme  and  involves  the  summation  over  x  and  y 
“patches”  of  elementary  ID  motion  detectors.  In  the 
fly  this  is  plausible,  given  the  known  sununation  proper¬ 
ties  of  specific  wide  field  lobula  plate  cells®.  Our  ID- ID 
scheme  on  the  other  hand  would  require  a  summation 
over  the  x  dimension  only  (in  our  example)  but  an  ori¬ 
ented  filtering  of  the  image  with  receptive  fields  elon¬ 
gated  in  y  befort  the  elementary  motion  detectors.  It 
is  possible  that  this  second  scheme  may  be  used  in  the 
fly  by  different  summation  cells  with  smaller  receptive 
fields.  It  is  ako  possible  that  the  wide  field  lobula  plate 
cells  effectively  implement  a  scheme  between  the  1D-2D 
and  the  ID-ID  by  using  some  oriented  filtering  before 
motion  detection  and  limited  y  integration  of  the  output 
of  the  elementary  motion  detectors.  Similar  considera¬ 
tions  may  apply  to  some  of  the  motion  selective  cortical 
cells. 

5.3  The  Time- to- Crash  detector 

The  TTC  detector  we  have  simulated  is  not  the  only 
possible  scheme.  Others  axe  possible  (see  for  instance 
[2])  that  take  into  account  more  complex  motions  than 
just  frontal  approach  to  a  horizontal  surface. 

It  is  also  conceivable  that  the  scheme  we  suggest  may 
be  simplified  even  further  in  certain  situations.  For  in¬ 
stance,  it  may  be  sufficient  in  the  summation  stage  to  use 
the  value  of  the  correlation  for  a  fixed  (and  reasonable) 
shift  -  instead  of  an  estimate  of  the  optical  flow,  that  is 
the  shift  that  maximize  correlation.  This  is  equivalent 
to  use  directly  the  output  of  Reichardt’s  correlation  nets 
instead  of  using  the  result  of  a  winner-take-all  operation 
on  a  set  of  Reichardt’s  nets  with  different  spacings  (or 
delays). 

Another  related  idea  is  to  continuously  adjust  the  cor¬ 
relation  shifts  in  order  to  track  as  closely  as  possible  the 
maximum  of  the  correlation  (or  the  minimum  of  the  dis¬ 
tance);  in  this  way  it  may  be  possible  to  reduce  the  com- 

®The  patch  would  be  very  large  and  would  correspond  to 
the  receptive  field  of  the  cell,  that  is  its  integration  donaain 


putation  of  the  correlation  to  just  a  few  shifts,  especially 
if  time-filtering  techniques  are  also  used. 
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Figure  10:  An  exjunple  of  optical  flows  computed  by  using  (a)  2D-2D,  (b)  1D-2D  and  (c)  ID-ID.  In  most  frame 
pairs  in  a  sequence  the  three  flows  are  much  more  similar  to  each  other. 
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Figure  1 1 :  TTC  measurements  comparison  by  using  2D-2D  and  1D-2D.  In  this  and  the  following  figures  the  abscissa 
gives  the  time  in  terms  of  elapsed  frames;  the  ordinate  gives  the  estimate  of  the  time  to  crash  in  frame  units. 
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Figure  14;  An  example  of  a  flow  field  computed  at  one  point  in  time  in  the  above  sequence  ,  obtained  by  using  (a) 
2D-2D  and  (b)  ID-ID. 


Figure  15:  Mean  (dotted  line)  and  standard  deviation  (dashed  line)  of  the  error  of  the  optical  flow  estimation. 
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Figure  16:  TTC  estimation  by  using  one  TTC  detector,  with  receptive  field  of  r  =  80  and  32  elementary  motion 
detectors.  The  slope  of  the  irut  TTC,  computed  by  using  the  optical  flows  obtained  by  2D~2D,  is  m  =  —0.672.  The 
slope  of  the  straight  line,  computed  by  using  the  TTC  measures  obtained  by  ID-ID,  is  m  =  -0.64.  A  comparison 
of  the  TTC  measures  obtained  by  1D~W  with  the  true  TTC  yields  a  mean  absolute  error  of  9.02,  with  a  standard 
deviation  of  9.54.  The  relative  error  in  the  mean  is  10.79%  with  a  standard  deviation  of  9.49%.  In  order  to  evaluate 
the  error  in  the  time  to  crash  estimation,  the  following  steps  have  been  performed.  The  true  time  to  crash  was 
estimated  from  a  linear  best  fit  of  the  TTC  measures  obtained  by  using  the  2D~2D  scheme  for  the  optical  flow 
estimation.  The  figures  show  the  straight  line  that  represents  the  theoretical  behavior  of  the  TTC.  A  linear  best 
fit  of  the  TTC  measures  obtained  by  using  the  ID-ID  scheme  for  the  optical  flow  estimation  was  then  performed 
in  order  to  evaluate  the  slopes  of  the  two  straight  lines.  The  absolute  and  relative  error  between  the  “true”  TTC 
and  the  one  measured  by  the  ID-ID  scheme  was  then  estimated.  Let  us  call  r*  the  true  TTC.  The  absolute  error  is 
Ea  =  \t*  —  r\  and  the  the  relative  error  is  Er  =  \t*  —  T|/|r|. 


Figure  17:  (a)  First  and  (b)  last  image  of  the  sequence. 
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Figure  20:  TTC  estimation  by  using  one  TTC  detector,  with  receptive  field  of  r  =  80  and  32  elementary  motion 
detectors.  The  slope  of  the  true  TTC,  computed  by  using  the  optical  flows  obtained  by  2D~2D,  is  m  =  -0.77.  The 
slope  of  the  straight  line,  computed  by  using  the  TTC  measures  obtained  by  1D~1D,  is  m  =  -0.83.  Comparing 
the  TTC  measures  obtained  by  ID-ID  with  the  true  TTC,  we  had  a  mean  absolute  error  of  8.02,  with  a  standard 
deviation  of  8.97.  With  respect  to  the  relative  error  we  had  a  mean  of  10.9%  and  a  standard  deviation  of  9.72%. 


Figure  21:  (a)  First  and  (b)  last  image  of  the  sequence. 
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Figure  23;  Mean  (dotted  line)  and  standard  deviation  (dashed  line)  of  the  error  relative  to  optical  flow  estimation. 
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Figure  24:  TTC  estimation  by  using  one  TTC  detector,  with  receptive  field  of  r  =  80  and  32  elementary  motion 
detectors.  The  slope  of  the  true  TTC,  computed  by  using  the  optical  flows  obtained  by  wta-2D,  is  m  =  —1.24.  The 
slope  of  the  straight  line,  computed  by  using  the  TTC  measures  obtained  by  ID-ID,  is  m  =  —1.14.  Comparing 
the  TTC  measures  obtained  by  ID-ID  with  the  true  TTC,  we  had  a  mean  absolute  error  of  7.6,  with  a  standard 
deviation  of  7.9.  With  respect  to  the  relative  error  we  had  a  mean  of  11.4%  and  a  standard  deviation  of  10.3%. 
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